A learning rule of neural networks via simultaneous perturbation and its hardware implementation

-This paper describes a learning rule of neural networks via a simultaneous perturbation and an analog feedforward neural network circuit using the learning rule. The learning rule used here is a stochastic gradient-like algorithm via a simultaneous perturbation. The learning rule requires only forward operations o f the neural network. Therefore, it is suitable for hardware implementation. First, we state the learning rule and show some computer simulation results o f the learning rule. A comparison between the learning rule, the usual back-propagation method, and a learning rule by a difference approximation is considered through the exclusive-OR problem and a simple pattern recognition problem known as the TCLX problem. Moreover, 26 alphabetical characters' recognition is handled to confirm a feasibility of the learning rule for large neural networks. Next, we describe details of the fabricated neural network circuit with learning ability. The exclusive-OR problem and the TCLX problem are considered. In a fabricated analog neural network circuit, input, output, and weights are realized by voltages. Keywords--Analog feedforward neural network circuit, Simultaneous perturbation, Learning rule, Hardware implementation.